Implementation of a variable step size backpropagation algorithm

This paper reports the effect of the step-size (learning rate parameter) on the performance of the backpropgation algorithm. Backpropagation algorithm (BP) is used to train multilayer neural network. BP algorithm is the generalized form of the least mean square (LMS) algorithm. In this proposed backpropagation algorithm different learning rate parameter are used in different layer. The learning rate parameter of a backpropagation algorithm is an important parameter as it determines the amount of correction applied as the network adapts from one iteration to the next. Choosing the appropriate step size is not always easy, usually requiring experience in neural network design. Hardware implementations of the proposed backpropagation algorithm are also shown in this work. For effective hardware utilizations artificial neural model is based on serial processing structure. Keywords—Neural network, Backpropagation algorithm, Least mean square algorithm, learning rate parameter.